CovidStanfordV1, Main, Exploration, bibRecord, 000846

A case study in model failure? COVID-19 daily deaths and ICU bed utilisation predictions in New York state.

Identifieur interne : 000846 ( Main/Exploration ); précédent : 000845; suivant : 000847

A case study in model failure? COVID-19 daily deaths and ICU bed utilisation predictions in New York state.

Auteurs : Vincent Chin [Australie] ; Noelle I. Samia [États-Unis] ; Roman Marchant [Australie] ; Ori Rosen [États-Unis] ; John P A. Ioannidis [États-Unis] ; Martin A. Tanner [États-Unis] ; Sally Cripps [Australie]

Source :

European journal of epidemiology [ 1573-7284 ] ; 2020.

RBID : pubmed:32780189

Descripteurs français

KwdFr :
- Betacoronavirus (MeSH), Humains (MeSH), Infections à coronavirus (mortalité), Modèles statistiques (MeSH), Mortalité (tendances), Pandémies (prévention et contrôle), Pneumopathie virale (mortalité), Prévision (méthodes), Santé publique (MeSH), Taux d'occupation des lits (MeSH), Unités de soins intensifs (ressources et distribution), Unités de soins intensifs (statistiques et données numériques), État de New York (épidémiologie).
MESH :
- mortalité : Infections à coronavirus, Pneumopathie virale.
- méthodes : Prévision.
- prévention et contrôle : Pandémies.
- ressources et distribution : Unités de soins intensifs.
- statistiques et données numériques : Unités de soins intensifs.
- tendances : Mortalité.
- épidémiologie : État de New York.
- Betacoronavirus, Humains, Modèles statistiques, Santé publique, Taux d'occupation des lits.

English descriptors

KwdEn :
- Bed Occupancy (MeSH), Betacoronavirus (MeSH), COVID-19 (MeSH), Coronavirus Infections (mortality), Forecasting (methods), Humans (MeSH), Intensive Care Units (statistics & numerical data), Intensive Care Units (supply & distribution), Models, Statistical (MeSH), Mortality (trends), New York (epidemiology), Pandemics (prevention & control), Pneumonia, Viral (mortality), Public Health (MeSH), SARS-CoV-2 (MeSH).
MESH :
- geographic , epidemiology : New York.
- methods : Forecasting.
- mortality : Coronavirus Infections, Pneumonia, Viral.
- prevention & control : Pandemics.
- statistics & numerical data : Intensive Care Units.
- supply & distribution : Intensive Care Units.
- trends : Mortality.
- Bed Occupancy, Betacoronavirus, COVID-19, Humans, Models, Statistical, Public Health, SARS-CoV-2.

Abstract

Forecasting models have been influential in shaping decision-making in the COVID-19 pandemic. However, there is concern that their predictions may have been misleading. Here, we dissect the predictions made by four models for the daily COVID-19 death counts between March 25 and June 5 in New York state, as well as the predictions of ICU bed utilisation made by the influential IHME model. We evaluated the accuracy of the point estimates and the accuracy of the uncertainty estimates of the model predictions. First, we compared the "ground truth" data sources on daily deaths against which these models were trained. Three different data sources were used by these models, and these had substantial differences in recorded daily death counts. Two additional data sources that we examined also provided different death counts per day. For accuracy of prediction, all models fared very poorly. Only 10.2% of the predictions fell within 10% of their training ground truth, irrespective of distance into the future. For accurate assessment of uncertainty, only one model matched relatively well the nominal 95% coverage, but that model did not start predictions until April 16, thus had no impact on early, major decisions. For ICU bed utilisation, the IHME model was highly inaccurate; the point estimates only started to match ground truth after the pandemic wave had started to wane. We conclude that trustworthy models require trustworthy input data to be trained upon. Moreover, models need to be subjected to prespecified real time performance tests, before their results are provided to policy makers and public health officials.

DOI: 10.1007/s10654-020-00669-6
PubMed: 32780189
PubMed Central: PMC7417851

Affiliations:

Links toward previous steps (curation, corpus...)

to stream Main, to step Corpus: 000439
to stream Main, to step Curation: 000439

Le document en format XML

<record><TEI><teiHeader><fileDesc><titleStmt><title xml:lang="en">A case study in model failure? COVID-19 daily deaths and ICU bed utilisation predictions in New York state.</title>
<author><name sortKey="Chin, Vincent" sort="Chin, Vincent" uniqKey="Chin V" first="Vincent" last="Chin">Vincent Chin</name>
<affiliation wicri:level="3"><nlm:affiliation>ARC Centre for Data Analytics for Resources and Environments, Sydney, Australia.</nlm:affiliation>
<country xml:lang="fr">Australie</country>
<wicri:regionArea>ARC Centre for Data Analytics for Resources and Environments, Sydney</wicri:regionArea>
<placeName><settlement type="city">Sydney</settlement>
<region type="état">Nouvelle-Galles du Sud</region>
</placeName>
</affiliation>
<affiliation wicri:level="4"><nlm:affiliation>School of Mathematics and Statistics, The University of Sydney, Sydney, Australia.</nlm:affiliation>
<country xml:lang="fr">Australie</country>
<wicri:regionArea>School of Mathematics and Statistics, The University of Sydney, Sydney</wicri:regionArea>
<placeName><settlement type="city">Sydney</settlement>
<region type="état">Nouvelle-Galles du Sud</region>
</placeName>
<orgName type="university">Université de Sydney</orgName>
</affiliation>
</author>
<author><name sortKey="Samia, Noelle I" sort="Samia, Noelle I" uniqKey="Samia N" first="Noelle I" last="Samia">Noelle I. Samia</name>
<affiliation wicri:level="3"><nlm:affiliation>Department of Statistics, Northwestern University, Chicago, USA.</nlm:affiliation>
<country xml:lang="fr">États-Unis</country>
<wicri:regionArea>Department of Statistics, Northwestern University, Chicago</wicri:regionArea>
<placeName><settlement type="city">Chicago</settlement>
<region type="state">Illinois</region>
</placeName>
</affiliation>
</author>
<author><name sortKey="Marchant, Roman" sort="Marchant, Roman" uniqKey="Marchant R" first="Roman" last="Marchant">Roman Marchant</name>
<affiliation wicri:level="3"><nlm:affiliation>ARC Centre for Data Analytics for Resources and Environments, Sydney, Australia.</nlm:affiliation>
<country xml:lang="fr">Australie</country>
<wicri:regionArea>ARC Centre for Data Analytics for Resources and Environments, Sydney</wicri:regionArea>
<placeName><settlement type="city">Sydney</settlement>
<region type="état">Nouvelle-Galles du Sud</region>
</placeName>
</affiliation>
<affiliation wicri:level="4"><nlm:affiliation>School of Mathematics and Statistics, The University of Sydney, Sydney, Australia.</nlm:affiliation>
<country xml:lang="fr">Australie</country>
<wicri:regionArea>School of Mathematics and Statistics, The University of Sydney, Sydney</wicri:regionArea>
<placeName><settlement type="city">Sydney</settlement>
<region type="état">Nouvelle-Galles du Sud</region>
</placeName>
<orgName type="university">Université de Sydney</orgName>
</affiliation>
</author>
<author><name sortKey="Rosen, Ori" sort="Rosen, Ori" uniqKey="Rosen O" first="Ori" last="Rosen">Ori Rosen</name>
<affiliation wicri:level="1"><nlm:affiliation>Department of Mathematical Sciences, University of Texas at El Paso, El Paso, USA.</nlm:affiliation>
<country xml:lang="fr">États-Unis</country>
<wicri:regionArea>Department of Mathematical Sciences, University of Texas at El Paso, El Paso</wicri:regionArea>
<wicri:noRegion>El Paso</wicri:noRegion>
</affiliation>
</author>
<author><name sortKey="Ioannidis, John P A" sort="Ioannidis, John P A" uniqKey="Ioannidis J" first="John P A" last="Ioannidis">John P A. Ioannidis</name>
<affiliation wicri:level="3"><nlm:affiliation>Stanford Prevention Research Center, Stanford, USA.</nlm:affiliation>
<country xml:lang="fr">États-Unis</country>
<wicri:regionArea>Stanford Prevention Research Center, Stanford</wicri:regionArea>
<placeName><settlement type="city">Stanford (Californie)</settlement>
<region type="state">Californie</region>
</placeName>
</affiliation>
<affiliation wicri:level="4"><nlm:affiliation>Department of Medicine, Stanford University, Stanford, USA.</nlm:affiliation>
<country xml:lang="fr">États-Unis</country>
<wicri:regionArea>Department of Medicine, Stanford University, Stanford</wicri:regionArea>
<placeName><settlement type="city">Stanford (Californie)</settlement>
<region type="state">Californie</region>
</placeName>
<orgName type="university">Université Stanford</orgName>
</affiliation>
<affiliation wicri:level="4"><nlm:affiliation>Department of Epidemiology and Population Health, Stanford University, Stanford, USA.</nlm:affiliation>
<country xml:lang="fr">États-Unis</country>
<wicri:regionArea>Department of Epidemiology and Population Health, Stanford University, Stanford</wicri:regionArea>
<placeName><settlement type="city">Stanford (Californie)</settlement>
<region type="state">Californie</region>
</placeName>
<orgName type="university">Université Stanford</orgName>
</affiliation>
<affiliation wicri:level="4"><nlm:affiliation>Department of Biomedical Data Sciences, Stanford University, Stanford, USA.</nlm:affiliation>
<country xml:lang="fr">États-Unis</country>
<wicri:regionArea>Department of Biomedical Data Sciences, Stanford University, Stanford</wicri:regionArea>
<placeName><settlement type="city">Stanford (Californie)</settlement>
<region type="state">Californie</region>
</placeName>
<orgName type="university">Université Stanford</orgName>
</affiliation>
<affiliation wicri:level="4"><nlm:affiliation>Department of Statistics, Stanford University, Stanford, USA.</nlm:affiliation>
<country xml:lang="fr">États-Unis</country>
<wicri:regionArea>Department of Statistics, Stanford University, Stanford</wicri:regionArea>
<placeName><settlement type="city">Stanford (Californie)</settlement>
<region type="state">Californie</region>
</placeName>
<orgName type="university">Université Stanford</orgName>
</affiliation>
<affiliation wicri:level="4"><nlm:affiliation>Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, USA.</nlm:affiliation>
<country xml:lang="fr">États-Unis</country>
<wicri:regionArea>Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford</wicri:regionArea>
<placeName><settlement type="city">Stanford (Californie)</settlement>
<region type="state">Californie</region>
</placeName>
<orgName type="university">Université Stanford</orgName>
</affiliation>
</author>
<author><name sortKey="Tanner, Martin A" sort="Tanner, Martin A" uniqKey="Tanner M" first="Martin A" last="Tanner">Martin A. Tanner</name>
<affiliation wicri:level="3"><nlm:affiliation>Department of Statistics, Northwestern University, Chicago, USA.</nlm:affiliation>
<country xml:lang="fr">États-Unis</country>
<wicri:regionArea>Department of Statistics, Northwestern University, Chicago</wicri:regionArea>
<placeName><settlement type="city">Chicago</settlement>
<region type="state">Illinois</region>
</placeName>
</affiliation>
</author>
<author><name sortKey="Cripps, Sally" sort="Cripps, Sally" uniqKey="Cripps S" first="Sally" last="Cripps">Sally Cripps</name>
<affiliation wicri:level="3"><nlm:affiliation>ARC Centre for Data Analytics for Resources and Environments, Sydney, Australia. sally.cripps@sydney.edu.au.</nlm:affiliation>
<country xml:lang="fr">Australie</country>
<wicri:regionArea>ARC Centre for Data Analytics for Resources and Environments, Sydney</wicri:regionArea>
<placeName><settlement type="city">Sydney</settlement>
<region type="état">Nouvelle-Galles du Sud</region>
</placeName>
</affiliation>
<affiliation wicri:level="4"><nlm:affiliation>School of Mathematics and Statistics, The University of Sydney, Sydney, Australia. sally.cripps@sydney.edu.au.</nlm:affiliation>
<country xml:lang="fr">Australie</country>
<wicri:regionArea>School of Mathematics and Statistics, The University of Sydney, Sydney</wicri:regionArea>
<placeName><settlement type="city">Sydney</settlement>
<region type="état">Nouvelle-Galles du Sud</region>
</placeName>
<orgName type="university">Université de Sydney</orgName>
</affiliation>
</author>
</titleStmt>
<publicationStmt><idno type="wicri:source">PubMed</idno>
<date when="2020">2020</date>
<idno type="RBID">pubmed:32780189</idno>
<idno type="pmid">32780189</idno>
<idno type="doi">10.1007/s10654-020-00669-6</idno>
<idno type="pmc">PMC7417851</idno>
<idno type="wicri:Area/Main/Corpus">000439</idno>
<idno type="wicri:explorRef" wicri:stream="Main" wicri:step="Corpus" wicri:corpus="PubMed">000439</idno>
<idno type="wicri:Area/Main/Curation">000439</idno>
<idno type="wicri:explorRef" wicri:stream="Main" wicri:step="Curation">000439</idno>
<idno type="wicri:Area/Main/Exploration">000439</idno>
</publicationStmt>
<sourceDesc><biblStruct><analytic><title xml:lang="en">A case study in model failure? COVID-19 daily deaths and ICU bed utilisation predictions in New York state.</title>
<author><name sortKey="Chin, Vincent" sort="Chin, Vincent" uniqKey="Chin V" first="Vincent" last="Chin">Vincent Chin</name>
<affiliation wicri:level="3"><nlm:affiliation>ARC Centre for Data Analytics for Resources and Environments, Sydney, Australia.</nlm:affiliation>
<country xml:lang="fr">Australie</country>
<wicri:regionArea>ARC Centre for Data Analytics for Resources and Environments, Sydney</wicri:regionArea>
<placeName><settlement type="city">Sydney</settlement>
<region type="état">Nouvelle-Galles du Sud</region>
</placeName>
</affiliation>
<affiliation wicri:level="4"><nlm:affiliation>School of Mathematics and Statistics, The University of Sydney, Sydney, Australia.</nlm:affiliation>
<country xml:lang="fr">Australie</country>
<wicri:regionArea>School of Mathematics and Statistics, The University of Sydney, Sydney</wicri:regionArea>
<placeName><settlement type="city">Sydney</settlement>
<region type="état">Nouvelle-Galles du Sud</region>
</placeName>
<orgName type="university">Université de Sydney</orgName>
</affiliation>
</author>
<author><name sortKey="Samia, Noelle I" sort="Samia, Noelle I" uniqKey="Samia N" first="Noelle I" last="Samia">Noelle I. Samia</name>
<affiliation wicri:level="3"><nlm:affiliation>Department of Statistics, Northwestern University, Chicago, USA.</nlm:affiliation>
<country xml:lang="fr">États-Unis</country>
<wicri:regionArea>Department of Statistics, Northwestern University, Chicago</wicri:regionArea>
<placeName><settlement type="city">Chicago</settlement>
<region type="state">Illinois</region>
</placeName>
</affiliation>
</author>
<author><name sortKey="Marchant, Roman" sort="Marchant, Roman" uniqKey="Marchant R" first="Roman" last="Marchant">Roman Marchant</name>
<affiliation wicri:level="3"><nlm:affiliation>ARC Centre for Data Analytics for Resources and Environments, Sydney, Australia.</nlm:affiliation>
<country xml:lang="fr">Australie</country>
<wicri:regionArea>ARC Centre for Data Analytics for Resources and Environments, Sydney</wicri:regionArea>
<placeName><settlement type="city">Sydney</settlement>
<region type="état">Nouvelle-Galles du Sud</region>
</placeName>
</affiliation>
<affiliation wicri:level="4"><nlm:affiliation>School of Mathematics and Statistics, The University of Sydney, Sydney, Australia.</nlm:affiliation>
<country xml:lang="fr">Australie</country>
<wicri:regionArea>School of Mathematics and Statistics, The University of Sydney, Sydney</wicri:regionArea>
<placeName><settlement type="city">Sydney</settlement>
<region type="état">Nouvelle-Galles du Sud</region>
</placeName>
<orgName type="university">Université de Sydney</orgName>
</affiliation>
</author>
<author><name sortKey="Rosen, Ori" sort="Rosen, Ori" uniqKey="Rosen O" first="Ori" last="Rosen">Ori Rosen</name>
<affiliation wicri:level="1"><nlm:affiliation>Department of Mathematical Sciences, University of Texas at El Paso, El Paso, USA.</nlm:affiliation>
<country xml:lang="fr">États-Unis</country>
<wicri:regionArea>Department of Mathematical Sciences, University of Texas at El Paso, El Paso</wicri:regionArea>
<wicri:noRegion>El Paso</wicri:noRegion>
</affiliation>
</author>
<author><name sortKey="Ioannidis, John P A" sort="Ioannidis, John P A" uniqKey="Ioannidis J" first="John P A" last="Ioannidis">John P A. Ioannidis</name>
<affiliation wicri:level="3"><nlm:affiliation>Stanford Prevention Research Center, Stanford, USA.</nlm:affiliation>
<country xml:lang="fr">États-Unis</country>
<wicri:regionArea>Stanford Prevention Research Center, Stanford</wicri:regionArea>
<placeName><settlement type="city">Stanford (Californie)</settlement>
<region type="state">Californie</region>
</placeName>
</affiliation>
<affiliation wicri:level="4"><nlm:affiliation>Department of Medicine, Stanford University, Stanford, USA.</nlm:affiliation>
<country xml:lang="fr">États-Unis</country>
<wicri:regionArea>Department of Medicine, Stanford University, Stanford</wicri:regionArea>
<placeName><settlement type="city">Stanford (Californie)</settlement>
<region type="state">Californie</region>
</placeName>
<orgName type="university">Université Stanford</orgName>
</affiliation>
<affiliation wicri:level="4"><nlm:affiliation>Department of Epidemiology and Population Health, Stanford University, Stanford, USA.</nlm:affiliation>
<country xml:lang="fr">États-Unis</country>
<wicri:regionArea>Department of Epidemiology and Population Health, Stanford University, Stanford</wicri:regionArea>
<placeName><settlement type="city">Stanford (Californie)</settlement>
<region type="state">Californie</region>
</placeName>
<orgName type="university">Université Stanford</orgName>
</affiliation>
<affiliation wicri:level="4"><nlm:affiliation>Department of Biomedical Data Sciences, Stanford University, Stanford, USA.</nlm:affiliation>
<country xml:lang="fr">États-Unis</country>
<wicri:regionArea>Department of Biomedical Data Sciences, Stanford University, Stanford</wicri:regionArea>
<placeName><settlement type="city">Stanford (Californie)</settlement>
<region type="state">Californie</region>
</placeName>
<orgName type="university">Université Stanford</orgName>
</affiliation>
<affiliation wicri:level="4"><nlm:affiliation>Department of Statistics, Stanford University, Stanford, USA.</nlm:affiliation>
<country xml:lang="fr">États-Unis</country>
<wicri:regionArea>Department of Statistics, Stanford University, Stanford</wicri:regionArea>
<placeName><settlement type="city">Stanford (Californie)</settlement>
<region type="state">Californie</region>
</placeName>
<orgName type="university">Université Stanford</orgName>
</affiliation>
<affiliation wicri:level="4"><nlm:affiliation>Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, USA.</nlm:affiliation>
<country xml:lang="fr">États-Unis</country>
<wicri:regionArea>Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford</wicri:regionArea>
<placeName><settlement type="city">Stanford (Californie)</settlement>
<region type="state">Californie</region>
</placeName>
<orgName type="university">Université Stanford</orgName>
</affiliation>
</author>
<author><name sortKey="Tanner, Martin A" sort="Tanner, Martin A" uniqKey="Tanner M" first="Martin A" last="Tanner">Martin A. Tanner</name>
<affiliation wicri:level="3"><nlm:affiliation>Department of Statistics, Northwestern University, Chicago, USA.</nlm:affiliation>
<country xml:lang="fr">États-Unis</country>
<wicri:regionArea>Department of Statistics, Northwestern University, Chicago</wicri:regionArea>
<placeName><settlement type="city">Chicago</settlement>
<region type="state">Illinois</region>
</placeName>
</affiliation>
</author>
<author><name sortKey="Cripps, Sally" sort="Cripps, Sally" uniqKey="Cripps S" first="Sally" last="Cripps">Sally Cripps</name>
<affiliation wicri:level="3"><nlm:affiliation>ARC Centre for Data Analytics for Resources and Environments, Sydney, Australia. sally.cripps@sydney.edu.au.</nlm:affiliation>
<country xml:lang="fr">Australie</country>
<wicri:regionArea>ARC Centre for Data Analytics for Resources and Environments, Sydney</wicri:regionArea>
<placeName><settlement type="city">Sydney</settlement>
<region type="état">Nouvelle-Galles du Sud</region>
</placeName>
</affiliation>
<affiliation wicri:level="4"><nlm:affiliation>School of Mathematics and Statistics, The University of Sydney, Sydney, Australia. sally.cripps@sydney.edu.au.</nlm:affiliation>
<country xml:lang="fr">Australie</country>
<wicri:regionArea>School of Mathematics and Statistics, The University of Sydney, Sydney</wicri:regionArea>
<placeName><settlement type="city">Sydney</settlement>
<region type="état">Nouvelle-Galles du Sud</region>
</placeName>
<orgName type="university">Université de Sydney</orgName>
</affiliation>
</author>
</analytic>
<series><title level="j">European journal of epidemiology</title>
<idno type="eISSN">1573-7284</idno>
<imprint><date when="2020" type="published">2020</date>
</imprint>
</series>
</biblStruct>
</sourceDesc>
</fileDesc>
<profileDesc><textClass><keywords scheme="KwdEn" xml:lang="en"><term>Bed Occupancy (MeSH)</term>
<term>Betacoronavirus (MeSH)</term>
<term>COVID-19 (MeSH)</term>
<term>Coronavirus Infections (mortality)</term>
<term>Forecasting (methods)</term>
<term>Humans (MeSH)</term>
<term>Intensive Care Units (statistics & numerical data)</term>
<term>Intensive Care Units (supply & distribution)</term>
<term>Models, Statistical (MeSH)</term>
<term>Mortality (trends)</term>
<term>New York (epidemiology)</term>
<term>Pandemics (prevention & control)</term>
<term>Pneumonia, Viral (mortality)</term>
<term>Public Health (MeSH)</term>
<term>SARS-CoV-2 (MeSH)</term>
</keywords>
<keywords scheme="KwdFr" xml:lang="fr"><term>Betacoronavirus (MeSH)</term>
<term>Humains (MeSH)</term>
<term>Infections à coronavirus (mortalité)</term>
<term>Modèles statistiques (MeSH)</term>
<term>Mortalité (tendances)</term>
<term>Pandémies (prévention et contrôle)</term>
<term>Pneumopathie virale (mortalité)</term>
<term>Prévision (méthodes)</term>
<term>Santé publique (MeSH)</term>
<term>Taux d'occupation des lits (MeSH)</term>
<term>Unités de soins intensifs (ressources et distribution)</term>
<term>Unités de soins intensifs (statistiques et données numériques)</term>
<term>État de New York (épidémiologie)</term>
</keywords>
<keywords scheme="MESH" type="geographic" qualifier="epidemiology" xml:lang="en"><term>New York</term>
</keywords>
<keywords scheme="MESH" qualifier="methods" xml:lang="en"><term>Forecasting</term>
</keywords>
<keywords scheme="MESH" qualifier="mortality" xml:lang="en"><term>Coronavirus Infections</term>
<term>Pneumonia, Viral</term>
</keywords>
<keywords scheme="MESH" qualifier="mortalité" xml:lang="fr"><term>Infections à coronavirus</term>
<term>Pneumopathie virale</term>
</keywords>
<keywords scheme="MESH" qualifier="méthodes" xml:lang="fr"><term>Prévision</term>
</keywords>
<keywords scheme="MESH" qualifier="prevention & control" xml:lang="en"><term>Pandemics</term>
</keywords>
<keywords scheme="MESH" qualifier="prévention et contrôle" xml:lang="fr"><term>Pandémies</term>
</keywords>
<keywords scheme="MESH" qualifier="ressources et distribution" xml:lang="fr"><term>Unités de soins intensifs</term>
</keywords>
<keywords scheme="MESH" qualifier="statistics & numerical data" xml:lang="en"><term>Intensive Care Units</term>
</keywords>
<keywords scheme="MESH" qualifier="statistiques et données numériques" xml:lang="fr"><term>Unités de soins intensifs</term>
</keywords>
<keywords scheme="MESH" qualifier="supply & distribution" xml:lang="en"><term>Intensive Care Units</term>
</keywords>
<keywords scheme="MESH" qualifier="tendances" xml:lang="fr"><term>Mortalité</term>
</keywords>
<keywords scheme="MESH" qualifier="trends" xml:lang="en"><term>Mortality</term>
</keywords>
<keywords scheme="MESH" qualifier="épidémiologie" xml:lang="fr"><term>État de New York</term>
</keywords>
<keywords scheme="MESH" xml:lang="en"><term>Bed Occupancy</term>
<term>Betacoronavirus</term>
<term>COVID-19</term>
<term>Humans</term>
<term>Models, Statistical</term>
<term>Public Health</term>
<term>SARS-CoV-2</term>
</keywords>
<keywords scheme="MESH" xml:lang="fr"><term>Betacoronavirus</term>
<term>Humains</term>
<term>Modèles statistiques</term>
<term>Santé publique</term>
<term>Taux d'occupation des lits</term>
</keywords>
</textClass>
</profileDesc>
</teiHeader>
<front><div type="abstract" xml:lang="en">Forecasting models have been influential in shaping decision-making in the COVID-19 pandemic. However, there is concern that their predictions may have been misleading. Here, we dissect the predictions made by four models for the daily COVID-19 death counts between March 25 and June 5 in New York state, as well as the predictions of ICU bed utilisation made by the influential IHME model. We evaluated the accuracy of the point estimates and the accuracy of the uncertainty estimates of the model predictions. First, we compared the "ground truth" data sources on daily deaths against which these models were trained. Three different data sources were used by these models, and these had substantial differences in recorded daily death counts. Two additional data sources that we examined also provided different death counts per day. For accuracy of prediction, all models fared very poorly. Only 10.2% of the predictions fell within 10% of their training ground truth, irrespective of distance into the future. For accurate assessment of uncertainty, only one model matched relatively well the nominal 95% coverage, but that model did not start predictions until April 16, thus had no impact on early, major decisions. For ICU bed utilisation, the IHME model was highly inaccurate; the point estimates only started to match ground truth after the pandemic wave had started to wane. We conclude that trustworthy models require trustworthy input data to be trained upon. Moreover, models need to be subjected to prespecified real time performance tests, before their results are provided to policy makers and public health officials.</div>
</front>
</TEI>
<pubmed><MedlineCitation Status="MEDLINE" Owner="NLM"><PMID Version="1">32780189</PMID>
<DateCompleted><Year>2020</Year>
<Month>09</Month>
<Day>16</Day>
</DateCompleted>
<DateRevised><Year>2021</Year>
<Month>01</Month>
<Day>10</Day>
</DateRevised>
<Article PubModel="Print-Electronic"><Journal><ISSN IssnType="Electronic">1573-7284</ISSN>
<JournalIssue CitedMedium="Internet"><Volume>35</Volume>
<Issue>8</Issue>
<PubDate><Year>2020</Year>
<Month>Aug</Month>
</PubDate>
</JournalIssue>
<Title>European journal of epidemiology</Title>
<ISOAbbreviation>Eur J Epidemiol</ISOAbbreviation>
</Journal>
<ArticleTitle>A case study in model failure? COVID-19 daily deaths and ICU bed utilisation predictions in New York state.</ArticleTitle>
<Pagination><MedlinePgn>733-742</MedlinePgn>
</Pagination>
<ELocationID EIdType="doi" ValidYN="Y">10.1007/s10654-020-00669-6</ELocationID>
<Abstract><AbstractText>Forecasting models have been influential in shaping decision-making in the COVID-19 pandemic. However, there is concern that their predictions may have been misleading. Here, we dissect the predictions made by four models for the daily COVID-19 death counts between March 25 and June 5 in New York state, as well as the predictions of ICU bed utilisation made by the influential IHME model. We evaluated the accuracy of the point estimates and the accuracy of the uncertainty estimates of the model predictions. First, we compared the "ground truth" data sources on daily deaths against which these models were trained. Three different data sources were used by these models, and these had substantial differences in recorded daily death counts. Two additional data sources that we examined also provided different death counts per day. For accuracy of prediction, all models fared very poorly. Only 10.2% of the predictions fell within 10% of their training ground truth, irrespective of distance into the future. For accurate assessment of uncertainty, only one model matched relatively well the nominal 95% coverage, but that model did not start predictions until April 16, thus had no impact on early, major decisions. For ICU bed utilisation, the IHME model was highly inaccurate; the point estimates only started to match ground truth after the pandemic wave had started to wane. We conclude that trustworthy models require trustworthy input data to be trained upon. Moreover, models need to be subjected to prespecified real time performance tests, before their results are provided to policy makers and public health officials.</AbstractText>
</Abstract>
<AuthorList CompleteYN="Y"><Author ValidYN="Y"><LastName>Chin</LastName>
<ForeName>Vincent</ForeName>
<Initials>V</Initials>
<AffiliationInfo><Affiliation>ARC Centre for Data Analytics for Resources and Environments, Sydney, Australia.</Affiliation>
</AffiliationInfo>
<AffiliationInfo><Affiliation>School of Mathematics and Statistics, The University of Sydney, Sydney, Australia.</Affiliation>
</AffiliationInfo>
</Author>
<Author ValidYN="Y"><LastName>Samia</LastName>
<ForeName>Noelle I</ForeName>
<Initials>NI</Initials>
<AffiliationInfo><Affiliation>Department of Statistics, Northwestern University, Chicago, USA.</Affiliation>
</AffiliationInfo>
</Author>
<Author ValidYN="Y"><LastName>Marchant</LastName>
<ForeName>Roman</ForeName>
<Initials>R</Initials>
<AffiliationInfo><Affiliation>ARC Centre for Data Analytics for Resources and Environments, Sydney, Australia.</Affiliation>
</AffiliationInfo>
<AffiliationInfo><Affiliation>School of Mathematics and Statistics, The University of Sydney, Sydney, Australia.</Affiliation>
</AffiliationInfo>
</Author>
<Author ValidYN="Y"><LastName>Rosen</LastName>
<ForeName>Ori</ForeName>
<Initials>O</Initials>
<AffiliationInfo><Affiliation>Department of Mathematical Sciences, University of Texas at El Paso, El Paso, USA.</Affiliation>
</AffiliationInfo>
</Author>
<Author ValidYN="Y"><LastName>Ioannidis</LastName>
<ForeName>John P A</ForeName>
<Initials>JPA</Initials>
<AffiliationInfo><Affiliation>Stanford Prevention Research Center, Stanford, USA.</Affiliation>
</AffiliationInfo>
<AffiliationInfo><Affiliation>Department of Medicine, Stanford University, Stanford, USA.</Affiliation>
</AffiliationInfo>
<AffiliationInfo><Affiliation>Department of Epidemiology and Population Health, Stanford University, Stanford, USA.</Affiliation>
</AffiliationInfo>
<AffiliationInfo><Affiliation>Department of Biomedical Data Sciences, Stanford University, Stanford, USA.</Affiliation>
</AffiliationInfo>
<AffiliationInfo><Affiliation>Department of Statistics, Stanford University, Stanford, USA.</Affiliation>
</AffiliationInfo>
<AffiliationInfo><Affiliation>Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, USA.</Affiliation>
</AffiliationInfo>
</Author>
<Author ValidYN="Y"><LastName>Tanner</LastName>
<ForeName>Martin A</ForeName>
<Initials>MA</Initials>
<AffiliationInfo><Affiliation>Department of Statistics, Northwestern University, Chicago, USA.</Affiliation>
</AffiliationInfo>
</Author>
<Author ValidYN="Y"><LastName>Cripps</LastName>
<ForeName>Sally</ForeName>
<Initials>S</Initials>
<Identifier Source="ORCID">http://orcid.org/0000-0003-3207-172X</Identifier>
<AffiliationInfo><Affiliation>ARC Centre for Data Analytics for Resources and Environments, Sydney, Australia. sally.cripps@sydney.edu.au.</Affiliation>
</AffiliationInfo>
<AffiliationInfo><Affiliation>School of Mathematics and Statistics, The University of Sydney, Sydney, Australia. sally.cripps@sydney.edu.au.</Affiliation>
</AffiliationInfo>
</Author>
</AuthorList>
<Language>eng</Language>
<PublicationTypeList><PublicationType UI="D016428">Journal Article</PublicationType>
</PublicationTypeList>
<ArticleDate DateType="Electronic"><Year>2020</Year>
<Month>08</Month>
<Day>11</Day>
</ArticleDate>
</Article>
<MedlineJournalInfo><Country>Netherlands</Country>
<MedlineTA>Eur J Epidemiol</MedlineTA>
<NlmUniqueID>8508062</NlmUniqueID>
<ISSNLinking>0393-2990</ISSNLinking>
</MedlineJournalInfo>
<CitationSubset>IM</CitationSubset>
<MeshHeadingList><MeshHeading><DescriptorName UI="D001509" MajorTopicYN="N">Bed Occupancy</DescriptorName>
</MeshHeading>
<MeshHeading><DescriptorName UI="D000073640" MajorTopicYN="N">Betacoronavirus</DescriptorName>
</MeshHeading>
<MeshHeading><DescriptorName UI="D000086382" MajorTopicYN="N">COVID-19</DescriptorName>
</MeshHeading>
<MeshHeading><DescriptorName UI="D018352" MajorTopicYN="N">Coronavirus Infections</DescriptorName>
<QualifierName UI="Q000401" MajorTopicYN="Y">mortality</QualifierName>
</MeshHeading>
<MeshHeading><DescriptorName UI="D005544" MajorTopicYN="N">Forecasting</DescriptorName>
<QualifierName UI="Q000379" MajorTopicYN="Y">methods</QualifierName>
</MeshHeading>
<MeshHeading><DescriptorName UI="D006801" MajorTopicYN="N">Humans</DescriptorName>
</MeshHeading>
<MeshHeading><DescriptorName UI="D007362" MajorTopicYN="N">Intensive Care Units</DescriptorName>
<QualifierName UI="Q000706" MajorTopicYN="Y">statistics & numerical data</QualifierName>
<QualifierName UI="Q000600" MajorTopicYN="N">supply & distribution</QualifierName>
</MeshHeading>
<MeshHeading><DescriptorName UI="D015233" MajorTopicYN="N">Models, Statistical</DescriptorName>
</MeshHeading>
<MeshHeading><DescriptorName UI="D009026" MajorTopicYN="N">Mortality</DescriptorName>
<QualifierName UI="Q000639" MajorTopicYN="N">trends</QualifierName>
</MeshHeading>
<MeshHeading><DescriptorName UI="D009518" MajorTopicYN="N" Type="Geographic">New York</DescriptorName>
<QualifierName UI="Q000453" MajorTopicYN="N">epidemiology</QualifierName>
</MeshHeading>
<MeshHeading><DescriptorName UI="D058873" MajorTopicYN="N">Pandemics</DescriptorName>
<QualifierName UI="Q000517" MajorTopicYN="Y">prevention & control</QualifierName>
</MeshHeading>
<MeshHeading><DescriptorName UI="D011024" MajorTopicYN="N">Pneumonia, Viral</DescriptorName>
<QualifierName UI="Q000401" MajorTopicYN="Y">mortality</QualifierName>
</MeshHeading>
<MeshHeading><DescriptorName UI="D011634" MajorTopicYN="N">Public Health</DescriptorName>
</MeshHeading>
<MeshHeading><DescriptorName UI="D000086402" MajorTopicYN="N">SARS-CoV-2</DescriptorName>
</MeshHeading>
</MeshHeadingList>
<KeywordList Owner="NOTNLM"><Keyword MajorTopicYN="N">COVID-19</Keyword>
<Keyword MajorTopicYN="N">Hospital resource utilisation</Keyword>
<Keyword MajorTopicYN="N">Model evaluation</Keyword>
<Keyword MajorTopicYN="N">Uncertainty quantification</Keyword>
</KeywordList>
</MedlineCitation>
<PubmedData><History><PubMedPubDate PubStatus="received"><Year>2020</Year>
<Month>06</Month>
<Day>14</Day>
</PubMedPubDate>
<PubMedPubDate PubStatus="accepted"><Year>2020</Year>
<Month>07</Month>
<Day>21</Day>
</PubMedPubDate>
<PubMedPubDate PubStatus="pubmed"><Year>2020</Year>
<Month>8</Month>
<Day>12</Day>
<Hour>6</Hour>
<Minute>0</Minute>
</PubMedPubDate>
<PubMedPubDate PubStatus="medline"><Year>2020</Year>
<Month>9</Month>
<Day>17</Day>
<Hour>6</Hour>
<Minute>0</Minute>
</PubMedPubDate>
<PubMedPubDate PubStatus="entrez"><Year>2020</Year>
<Month>8</Month>
<Day>12</Day>
<Hour>6</Hour>
<Minute>0</Minute>
</PubMedPubDate>
</History>
<PublicationStatus>ppublish</PublicationStatus>
<ArticleIdList><ArticleId IdType="pubmed">32780189</ArticleId>
<ArticleId IdType="doi">10.1007/s10654-020-00669-6</ArticleId>
<ArticleId IdType="pii">10.1007/s10654-020-00669-6</ArticleId>
<ArticleId IdType="pmc">PMC7417851</ArticleId>
</ArticleIdList>
<ReferenceList><Reference><Citation>N Engl J Med. 2020 Jul 2;383(1):88-89</Citation>
<ArticleIdList><ArticleId IdType="pubmed">32343497</ArticleId>
</ArticleIdList>
</Reference>
<Reference><Citation>JAMA. 2020 May 19;323(19):1893-1894</Citation>
<ArticleIdList><ArticleId IdType="pubmed">32297897</ArticleId>
</ArticleIdList>
</Reference>
<Reference><Citation>N Engl J Med. 2020 Jul 23;383(4):303-305</Citation>
<ArticleIdList><ArticleId IdType="pubmed">32412711</ArticleId>
</ArticleIdList>
</Reference>
<Reference><Citation>J Am Geriatr Soc. 2020 Aug;68(8):1653-1656</Citation>
<ArticleIdList><ArticleId IdType="pubmed">32484912</ArticleId>
</ArticleIdList>
</Reference>
<Reference><Citation>Ann Oncol. 2020 Aug;31(8):1065-1074</Citation>
<ArticleIdList><ArticleId IdType="pubmed">32442581</ArticleId>
</ArticleIdList>
</Reference>
<Reference><Citation>Eur J Clin Invest. 2020 Apr;50(4):e13222</Citation>
<ArticleIdList><ArticleId IdType="pubmed">32191341</ArticleId>
</ArticleIdList>
</Reference>
</ReferenceList>
</PubmedData>
</pubmed>
<affiliations><list><country><li>Australie</li>
<li>États-Unis</li>
</country>
<region><li>Californie</li>
<li>Illinois</li>
<li>Nouvelle-Galles du Sud</li>
</region>
<settlement><li>Chicago</li>
<li>Stanford (Californie)</li>
<li>Sydney</li>
</settlement>
<orgName><li>Université Stanford</li>
<li>Université de Sydney</li>
</orgName>
</list>
<tree><country name="Australie"><region name="Nouvelle-Galles du Sud"><name sortKey="Chin, Vincent" sort="Chin, Vincent" uniqKey="Chin V" first="Vincent" last="Chin">Vincent Chin</name>
</region>
<name sortKey="Chin, Vincent" sort="Chin, Vincent" uniqKey="Chin V" first="Vincent" last="Chin">Vincent Chin</name>
<name sortKey="Cripps, Sally" sort="Cripps, Sally" uniqKey="Cripps S" first="Sally" last="Cripps">Sally Cripps</name>
<name sortKey="Cripps, Sally" sort="Cripps, Sally" uniqKey="Cripps S" first="Sally" last="Cripps">Sally Cripps</name>
<name sortKey="Marchant, Roman" sort="Marchant, Roman" uniqKey="Marchant R" first="Roman" last="Marchant">Roman Marchant</name>
<name sortKey="Marchant, Roman" sort="Marchant, Roman" uniqKey="Marchant R" first="Roman" last="Marchant">Roman Marchant</name>
</country>
<country name="États-Unis"><region name="Illinois"><name sortKey="Samia, Noelle I" sort="Samia, Noelle I" uniqKey="Samia N" first="Noelle I" last="Samia">Noelle I. Samia</name>
</region>
<name sortKey="Ioannidis, John P A" sort="Ioannidis, John P A" uniqKey="Ioannidis J" first="John P A" last="Ioannidis">John P A. Ioannidis</name>
<name sortKey="Ioannidis, John P A" sort="Ioannidis, John P A" uniqKey="Ioannidis J" first="John P A" last="Ioannidis">John P A. Ioannidis</name>
<name sortKey="Ioannidis, John P A" sort="Ioannidis, John P A" uniqKey="Ioannidis J" first="John P A" last="Ioannidis">John P A. Ioannidis</name>
<name sortKey="Ioannidis, John P A" sort="Ioannidis, John P A" uniqKey="Ioannidis J" first="John P A" last="Ioannidis">John P A. Ioannidis</name>
<name sortKey="Ioannidis, John P A" sort="Ioannidis, John P A" uniqKey="Ioannidis J" first="John P A" last="Ioannidis">John P A. Ioannidis</name>
<name sortKey="Ioannidis, John P A" sort="Ioannidis, John P A" uniqKey="Ioannidis J" first="John P A" last="Ioannidis">John P A. Ioannidis</name>
<name sortKey="Rosen, Ori" sort="Rosen, Ori" uniqKey="Rosen O" first="Ori" last="Rosen">Ori Rosen</name>
<name sortKey="Tanner, Martin A" sort="Tanner, Martin A" uniqKey="Tanner M" first="Martin A" last="Tanner">Martin A. Tanner</name>
</country>
</tree>
</affiliations>
</record>

Pour manipuler ce document sous Unix (Dilib)

EXPLOR_STEP=$WICRI_ROOT/Sante/explor/CovidStanfordV1/Data/Main/Exploration

HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 000846 | SxmlIndent | more

HfdSelect -h $EXPLOR_AREA/Data/Main/Exploration/biblio.hfd -nk 000846 | SxmlIndent | more

Pour mettre un lien sur cette page dans le réseau Wicri

{{Explor lien
   |wiki=    Sante
   |area=    CovidStanfordV1
   |flux=    Main
   |étape=   Exploration
   |type=    RBID
   |clé=     pubmed:32780189
   |texte=   A case study in model failure? COVID-19 daily deaths and ICU bed utilisation predictions in New York state.
}}

Pour générer des pages wiki

HfdIndexSelect -h $EXPLOR_AREA/Data/Main/Exploration/RBID.i   -Sk "pubmed:32780189" \
       | HfdSelect -Kh $EXPLOR_AREA/Data/Main/Exploration/biblio.hfd   \
       | NlmPubMed2Wicri -a CovidStanfordV1

This area was generated with Dilib version V0.6.38.
Data generation: Tue Feb 2 21:24:25 2021. Site generation: Tue Feb 2 21:26:08 2021

	Serveur d'exploration sur le Covid à Stanford
	Attention, ce site est en cours de développement ! Attention, site généré par des moyens informatiques à partir de corpus bruts. Les informations ne sont donc pas validées.

Serveur d'exploration sur le Covid à Stanford

A case study in model failure? COVID-19 daily deaths and ICU bed utilisation predictions in New York state.

A case study in model failure? COVID-19 daily deaths and ICU bed utilisation predictions in New York state.

Source :

Descripteurs français

English descriptors

Abstract

Links toward previous steps (curation, corpus...)

Le document en format XML

Pour manipuler ce document sous Unix (Dilib)

Pour mettre un lien sur cette page dans le réseau Wicri

Pour générer des pages wiki